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ABSTRACT 



Increasing dropout rates, marked decline in college 



preparatory enrollment, and reports of students' growing 
dissatisfaction with school prompted this exploration of the 
dimensions underlying teaching and the ways that microeconomics might 
positively influence teaching effectiveness. The literature review 
supports 10 advantages offered by student evaluations of teaching 
(SETS) in complementing student achievement as a schooling 
effectiveness measure. The Fox SETS studies show that teachers' 
enthusiasm accounts for twice the student achievement that lecture 
content does. This study's analysis was based on a sample of 6U 
teacher aggregates of 3,961 student questionnaires gathered over 2 
years from the same high school. Factor analysis of 52 variables 
revealed "student-caring" versus "task-driven" as key research-based, 
teaching dimensions. The resulting "managerial grid" for teachers 
offered graphic perspectives of interrelationships not afforded by 
conventional frameworks. Using "student satisfaction with teacher 
performance" as the dependent variable, allocativa efficiency was 
probed using planar and other quantitative techniques. Results show 
that "student-caring" has substantially mora influence on 
satisfaction than does "task-driven." Teachers are implicitly paid 
considerably less for caring than for task-driven behaviors. 
Reallocating salaries to produce teaching that interests students 
poses formidable technical and political problems. However, more 
caring for students, not necessarily more money allocated to it, 
should also produce higher percentages of satisfied students. The 
managerial grid should help define and achieve more appropriate 
teaching styles. Included are 128 endnotes. (Author/MLH) 
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ABSTRACT 

PROBLEM/OBJECTIVES 

Precipitous increases in high school dropout rates ^ marked 
declines in college preparatory enrollment^ and reports that stu- 
dents are dissatisfied with school^ prompted this exploration of 
what dimensions underlie teaching and how microeconomics might 
help efficiently produce teaching that genuinely interests^ 
and challenges students. 

PERSPECTIVES/THEORETICAL FRAMEWORK 

The literature review finds support for ten advantages that 
student evaluations of teaching (SETs) offer in complementing stu- 
dent achievement as a schooling effectiveness measure* And 
reanalysis here of the Dr. Fox studies of SETs shows teacher's 
enthusiasm to not only stroncrly affect evaluations r but also to 
consistently account for over twice the student achievement that 
lecture content does. 

METHODS/DATA SOURCE 

This study's main analysis, however, was based on a sample of 
60 teacher aggregates of 3,961 student questionnaires gathered 
over two years from the same high school. Factor analysis of 52 
variables revealed "student-caring" versus "task-driven," as key 
teaching dimensions — dimensions that persist across fourteen cited 
management and psychology studies of other supervisory contexts. 
The resulting "managerial grid for teachers" offered graphic per- 
spectives of interrelationships not afforded by conventional 
frameworks . 

Using a proxy for "student satisfaction with teacher perfor- 
mance" as the dependent variable, allocative efficiency was probed 
with planar, quadratic, Cobb-Douglas, probit and logit models of 
educational production functions fitted to proxies for the two 
grid dimensions. Though the curved surface of the quadratic 
explained the most variation and reflected textbook-like diminish- 
ing returns along both dimensions, the planar model best satisfied 
constraints of fit, and simplicity. Adding teacher's salary 
determinants and controlling statistically for nonteacher 
influences produced substantially unequal numerators and denom- 
inators for the requisite marginal-product-to-marginal-price 
ratios. 

RESULTS/CONCLUSIONS 

According to microeconomic theory, these disparities imply 
that the high school observed does not efficiently allocate sal- 
aries to produce teaching that interests students. Student-caring 
has substantially more influence on satisfaction than does task- 
driven but teachers are implicitly paid considerably less for 
caring than for task-driven behaviors.^ Moreover, teacher's salary 
is explicitly driven most by teaching experience, and relatively 
less by education — neither of which relate to student perceptions 
of teacher performance (just as others, e.g., Hanushek, have found 
them not to relate to student achievement). 
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IMPORTANCE/POLICY IMPLICATIONS 

Reallocating salaries to efficiently produce teaching that 
interests students surely poses formidable technical and political 
problems, Xet the data support the intuitively unsurprising 
notion thaw more student-caring — not necessarily more money real- 
located to it — should also produce higher percentages of satisfied 
students . To that end^ changes are suggested involving taacher- 
training institutions^ hiring, career ladders, in-service enrich- 
ment programs, and teacher pay. 

Policy change responsive to this study is well within reach 
in California. For example, just as the California Assessment 
Program (CAP) gathers achievement measures statewide, so too — 
perhaps even on the same test instrument — could it gather SET data 
covering teacher traits and student satisfaction. Indeed, just as 
California monetarily rewards CAP achievement gains, so too might 
it reward improvements in the satisfaction and retention of stu- 
dents, 

SCIENTIFIC IMPORTANCE/FUTURE RESEARCH 

Future research is urged that uses the managerial grid as the 
floor of a three dimensional space to: 1 ) analyze more comprehen- 
sive models in olving interaction of achievement, satisfaction, 
teaching styles and other key schooling variables; 2) establish 
quantitatively, the output consequences of teaching style trade- 
offs; and, 3) articulate preferred teaching styles (as Blake et 
al, did in applying their grid to other professions). 

Above all, the grid should help define and achieve teaching 
styles that better mix student-caring/ task-driven inputs, and 
satisfaction/achievement outputs — to aid the bigger picture of 
long-term student wellbeing and reversal of trends that prompted 
this study, 

[Note: This work builds upon: earlier research done at Stanford 
University, funded by the Ford Foundation, Although I alone, am 
responsible for any errors or omissions, I remain especially 
grateful to Ed Bridges, Jay Chambers, Nathan L, Gage, Mike Caret, 
Michael Kirst, Henry Levin, Ray Perry, and others for their 
good ideas and feedback,] 
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1«1 PROBLEM 

In Aprils 1983^ the National Commission on Excellence in 
Education (NCEE) reported that the percentage of students taking 
"general track" (i.e.^ non-college preparatory and non- 
vocational) courses jumped from 12 percent in 1964 to 42 percent 
in 1979.' In Calif ornia^ the percentage of students who drop out 
between the ninth and twelfth grades soared from 12 percent in 
1970^ to 22 percent in 1979,2 This is consistent with an earlier 
report of the California Commission for Reform of Intermediate 
and Secondary Education (RISE Commission^ 1975) that stated 
"increasing numbers of young people find schooling boring and 
ineffective^ unable to challenge their abilities^ meet their 
goals ^ or prepare them for adulthood,"^ It also agrees with a 
1983 Carnegie Foundation study that states "Most seniors agree 
that there should be more emphasis on academic subjects and that 
poor teaching has interfered with their education • . . "^ 
Indeed r the rash of 1983 reports seem to agree that "teacher 
training should be revised; that there should be more emphasis on 
subject matter ... and ... better approaches to teaching 
methods."^ 

More recent reports have reaffirmed the above. The Carnegie 
Forum on Education and the Economy, states: 

Concern over the quality of education in this country has 
been expressed in repeated warnings from the Education 
Commission of the States' Task force on Education and 
economic Growth , the President's Commission on Industrial 
Competitiveness, the National Alliance of Business and 
others. . .Large numbers of American children are in limbo-- 
ignorant of the past and unprepared for the future. Many 
are dropping out — not just out of school but out of 
productive society... An economy based on people who think 
for a living requires schools dedicated to the creation of 
environments in which students become very adept at think 
for themselves, places where they master th^ art of 
learning and acquire a strong taste for it.^ 

If the above trends are as severe as reported, they may be 
due, in part, to the lack of any systematic plan to produce 
teaching that attracts, challenges, and fulfills students. More 
precisely, just as changing teachers' salary determinants could 
produce more student learning per dollar,' so too might a similar 
change yield more efficient production of teaching that engages 
and enlivens students. ° 

The RISE, NCEE, National Task Force on Education for Eco- 
nomic Growth (NTFEEG), and Carnegie reports, recommend changes 
consistent with such a plan; e.g., they urge that teacher's 
salaries^ and jobs'^ be linked to teacher performance; that 
learners be considered "the primary clients — the most important 
individuals — served by the schools";'' and, that staff evaluation 
procedures "/> . . allow for the consideration of representative 
views of those affected by the performance of the staff member 
under review."'*^ The Carnegie report adds that present 
incentives: 
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•••not only do not reward performance and productivity^ but 
sometimes actually discourage them, Americans already 
spend more per capita on education than any other country 
in the free worlds This plan will work only if those who 
must finance it are convinced that the funds are being used 
as efficiently as possible*''* 

How to use those funds as efficiently as possible? If a 
school district chose to follow the above recommendations in a 
strict economic sense^ that is^ if — subject to a given budget 
constraint — it wanted to produce a maximum of teaching that truly 
engages and enriches students — then it would first need to 
ascertain the requisite microeconomic conditions of efficiency. 
Defining "efficiency" is aided by Levin's distinctions between 
technical^ allocative^ and social efficiency^ where: 

1 • Technical efficiency refers to transforming a 

given combination of inputs into the maximum output 
(graphically depicted in two-input space by any com- 
bination of the two inputs on a given isoproduct 
curve) ; 

2. Allocative or "price" efficiency considers relative 
input prices to find what technically efficient com- 
bination of inputs produces the maximum output for a 
given budget (Graphically, this is where the budget 
line is tangent to the highest possible isoquant)'^; 
and, finally, 

3. Social efficiency means lifting society to its 

highest possible indifference curve by "the optimal" 

combination of outputs (that point in utility space 

where the highest possible social welfare function 

is tangent to the grand utility possibility fron- 
tier)/5 

While this study speaks directly to allocative or price 
efficiency (in allocating teachers' salaries to produce maximum 
student satisfaction), it also relates to views on social 
efficiency. More specifically, some argue that society would be 
better off caring less about the traditional products (e.g., 
achievement) of schools and more about how the schooling process 
affects students and teachers. In his "Study of Schooling," John 
Goodlad concludes: 

We can make our schools more efficient. But making 
them relevant in the lives of boys and girls is oae of 
the most demanding challenges we face.'^ 

In a similar vein, Jencks et al. argue: 

Instead of evaluating schools in terms of long-term 
effects on their alumni, which appear to be relatively 
uniform, we think it wiser to evaluate schools in terms 
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of their immediate effects on teachers and students^ 
which appear much more variable. Some schools are dull^ 
depressing r even terrifying places , while others are 
lively^ comfortable r and reassuring. If we think of 
school life as an end in itself rather than a means to 
some other end^ such differences would not do much to 
make adults more equal, but it would do a great deal to 
make the quality of children's (and teachers') lives 
more equal. Since children are in school for a fifth 
of their lives, this would be a significant accom- 
plishment.^ ' 

Some prefer this focus even if it results in a classical product 
transformation curve — where gains in student satisfaction are 
traded for declines in student achievement. Indeed, A.S. Neil 
writes that he would rather see schools produce happy street 
cleaners than neurotic scholars. '° Though related to Neil's 
view, the bulk of this study will be considerably less 
philosophical, focusing largely on the technical and empirical 
issui^ 3 surrounding the allocative e fficiency of producing 
schoC'lng that interests students.*"^ 

1.2 QUESTIONS 

The challenge of allocating all resources to efficiently 
produce satisfied students is simplified by seeing that teacher's 
salaries and student satisfaction with teaching cover most of the 
bigger picture linking overall district budgets to student 
satisfaction with school in general . This follows because 
teachers' salaries consume approximately 70 percent of those 
budgets; and, roughly 80 percent of the student's time in school 
is spent with teachers. 

Measuring satisfaction becomes empirically feasible by 
ascertaining student perceptions of, and reactions to, teacher 
behaviors via "Student Evaluation of Teacher'^ (SET) 
questionnaires. 

From the above perspective, this study attempts to answer 
the following: ^ 

1 . What advantages might student evaluations of teachers 
offer California public high schools as an added mea- 
sure of schooling effectiveness? 

2. What fundamental dimensions of teaching underlie such 
evaluations? 

3. To what extent does one public high school in Califor- 
nia already satisfy the microeconomic conditions to 
efficiently produce teaching that interests students? 
That is, how much impact do specific teacher traits 
(e.g., knowledge of the subject matter, preparation for 
class, enthusiasm) have on overall student satisfac- 
tion? What implicit monetary prices do districts pay 
for those traits? To what extent are the impact-to- 
price ratios disparate across those traits? 
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4. To what specific inefficiencies do any disparities 
point and what policy changes do they imply? 

2*1 PROBLEM CONTEXT^O 

Efficiently translating dollars into desired educational 
outcomes has long been a concern of econoirists. Writing over two 
centuries ago in his Wealth of Nations » Adam Smith urged that the 
school master be "partly^ but not wholly paid by the public; 
because if he was wholly ^ or even principally paid by it^ he 
would soon learn to neglect his business."^' Such pessimism 
about the effects of unsound incentives in schools has been 
persistent. Indeed^ economists' claims of misguided incentives 
inducina inefficiencies in today's schools^ are widespread,"^^ 
24 25 26 Most recently^ Hanushek asserts: "The incentives 
existing within schools appear to provide a compelling 
explanation of public school inefficiencies" ' and "• • • the 
current structure of public education offers little hope for self 
correction."^ Levin articulates the broader problems: 

The conditions under which educational expenditures 
will translate into improved educational outcomes are 
fairly stringent. Firsts there must be substantial 
agreement on which outcomes are important; second , 
there must be knowledge of how added resources can be 
used to improve those outcomes; and third, those who 
are responsible for the educational process must have 
incentives to maximize the socially desirable outcomes. 
In each case, the actual situation seems to contrast 
with the assumptions that are necessary for added dol- 
lars to improve educational results • • • There is no 
agreement on educational priorities , no body of know- 
ledge that can predict the effects of different school 
policies on educational outcomes, and no relation bet- 
ween •staff incentives and the rhetorical goals of 
schools. This all suggests that school budgets are not 
likely to be translated efficiently into improved edu- 
cational outcomes. Rather, the actual application cf 
the increased support is likely to be determined by the 
power and interests of the decision makers them- 
selves • • 

Important among the interests of decision makers — student 
achievement aside--is maintaining smooth political sailing among 
the adults with whom they work. ° 

Hanushek feels that evaluation difficulties further 
influence these interests at the expense of both student and 
community interests: 

• • • The fact that school policies are so hard to 
evaluate makes it easier for teachers and school offi- 
cials to be influenced by their personal interests • • • 
Without direct evaluations, teachers and school offi- 
cials need not confront the possibility that their pri- 
vately preferred policies may be useless for the student 
and bad for the community ."^^ 
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Though school administrators* reluctance to evaluate 
teachers results^ in part^ from technical difficulties in 
measuring schooling outcomes ^ it also follows from the basic 
structural characteristics of the public school setting; i.e.r 
service to captive clientele in a non-competitive^ non-profit 
environment that yields no rewards to decision makers for 
efficiency in production. In discussing the corresponding 
administrator strategies ^ Chambers argues: 

Since they are unable to appropriate any pecuniary bene^ 
fits to themselves and since they generally operate sub- 
ject only to a minimal survival mechanism^ school deci- 
sion-makers find that the cost of inefficient behavior is 
relatively low. Their efforts , however^ ?re directed 
toward the reduction of the psychic costs involved in the 
management of district operations. Based upon these moti- 
vations ^ there is little incentive for school administra- 
tors to obtain precise evaluations of performance of the 
school system. Any such evaluation of the outputs of the 
system could only serve to reveal possible deficiencies 
in their own managerial abilities and, thusr threaten 
their survival* The more abstruse the method of evalua- 
tion^ the greater reliance the school board and the 
community must Pl^g^ in the judgment of these profes- 
sional educators. 

According to Hanushek^ administrators are likely to go 
beyond mere abstrusive evaluation to help teachers because both 
groups' interests often coincide: 

Administrators r who are typically drawn from the ranks of 
teachers r share many conceptions about what is "right." 
Moreover^ it generally serves the interest of admini- 
strators to accept teachers* arguments about reducing 
class sizesr raising salaries, and increasing expendi- 
tures: such policies increase the administrators' domain? 
lessen conflicts with their employees r and ultimately 
must affect their own salaries.**^ 

These and other coincidental interests, and the absence of 
rewards to educators to improve schooling, heighten the chances 
for interests of the deciders to encroach on those of children. 
Moreover, any excellence that emerges is likely to flow more 
despite such a system than because of it; i.e., likely to flow 
more from personal goals and drive within especially motivated 
individuals » than from organizational incentives external to 



them.-^^ 

2«2 ALLOCATION OF T£ACH£RS* SALARIES 

The widespread time-honored system of paying teachers 
according to years and units, rather than performance, clearly 
favors administrative convenience and political tranquillity over 
incentives for efficiency. Years and units are administratively 
easy to measure and translate into higher salaries. The process 




ERLC 



9 



' HANA6ERIAL GRID FOR TEACHERS, Foote, 2/2S/88 



PAGE 9 



requires no consideration of substantial inter- job differences in 
supply and demand ^ workloads ^ or required competencies (despite 
the high costs of ignoring these differences ) • Above all^ it 
requires no periodic^ conflict-producing evaluations of teachers* 
Indeed r the process is "automatic" in the senje that each year^ 
"action" is taken on salaries — they increase regardless of 
classroom performance--with little effort or political sacrifice 
from administrators. 

Although the above salary oractice is nearly universal 
throughout our public schools ^ a consistent body of research 
shows tbat it does not efficiently produce student achievement.^ 
42 43 44 Indeed r Hanushek found no relationship whatsoever 
between student achievement and teacher's years of service or 
post B.A. units: 

♦ . • the present set of hiring practices leads to 
an inefficient allocation of resources. The analysis 
indicates that teaching experience and graduate educa- 
tion do not contribute to gains in student achievement 
scores. Moreover^ the characteristics that do matter 
are not highly correlated with these factors. Yet these 
attributes are being purchased by the school district. 
Since turnover is costly , some average experience level 
over one year would be reasonable. However, the cur- 
rent average of over eleven years is certainly exces- 
sive*^ 

After reviewing a far wider variety of inputs among 130 separate 
studies, Hanushek adds: "Within the range of current school 
operations r variations in expenditures or in any other commonly 
identified determinants of school "quality" bear no systematic 
relationship to variations in the performance of students. "^^ 

Though certain studies show teacher experience to none- 
theless correlate positively with achievement, ' the 
research of Keeler and McCall suggests that rather than high 
experience causing high achievement, "the major reason for the 
relation between experience and achievement is the ability of 
experienced teachers to get into the schools of the high achiev- 
ers."^^ Because most districts' teachers are concentrated near 
the high ends of the years and units scales, the cost of these 
inputs represents a significant and periodically increasing 
national^ ' expense. 

The portion of this expense that serves to retain teachers 
can be justified because it reduces the costs of recruiting and 
training. T^d relative to a more competitive performance-based 
pay system, where older teachers might refrain from helping 
younger ones, the portion of years and units costs that encourage 
teamwork between old and young can also be justified. Yet 
whatever the magnitude of these, and perhaps other, justifiable 
portions, it is likely that their total falls short of the 
entirety spent on these inputs nationally; especially with years 
and units often accounting for more than half of a district s 
maximum teacher salary, and teacher's salaries consuming 
between 60 and 80 percent of districts' total operating budgets. 
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2*3 BETTER ALLOCATION CRITERIA? 

If we reject the current pay criteria because the evidence 
suggests it is inefficient ^ what better criteria are there? 
Hanushek's review of 130 input/output studies,, mentioned earlier^ 
reveals studies that do^ and others that do not^ show 
statistically significant effects of teacher inputs on student 
performance. Though these results suggest that no input bears 
any nationally generalizable relationship to student performance^ 
this does not preclude that any one study might have accurately 
measured effects that work well for the particular students 
sampled. For example^ Levin founds via linear^^ regression 
coefficients^ that reallocating teachers' salaries according to 
teachers' verbal scores could yield up to ten times the student 
verbal achievement per dollar that present allocation to 
teachers' experience does.^^ And a later work by Levin showed 
teacher's verbal score to be four times as potent as teacher 
experience Though the Levin studies suggest that teacher's 
verbal score might be a superior input , neither study is 
sufficiently definitive to warrant widespread policy change. 

While others argue that competition would force reallocation 
to more efficient inputs ^ this study tackles schools' 

incentives^ evaluation^ and salary allocation problems within the 
existing non-competitive setting; by incorporating consumer 
(student) preferences into a microeconomic analysis of those 
problems. More precisely , just as paying teachers according to 
years and units appears inconsistent with efficiently producing 
student achievement ^ so too might it be inconsistent with effi- 
ciently producing teaching that attracts students. The lack of 
any agreement on even the sign of the correlation between tea- 
cher's years experience and student ratings of teaching guality 
(e.g.^ across the many studies reviewed by Costin et al.^^) sug- 
gests this might be so. And just as Levin found an input that 
seems to produce learning more efficiently among a given student 
sample^ so too might a similar methodology reveal inputs that 
produce satisfaction more efficiently. 

Such a methodology would require vneasuring students* 
perceptions of r and reactions to, specific teacher traits. This 
is commonly done with student evaluations of teaching 
questionnaires (SETs). Such an effort assumes that SET results 
would offer valuable information not afforded by conventional 
achievement measures — an assumption explored in the following 
section. 

2.4 SETS AS A COMPLEMENTARY MEASURE OF SCHOOLING OUTPUT 
The use of SETs as one measure of teacher output is 
widespread at the college level"^->-and scant among high schools. 
This may result^ in part^ from there being more serious focus on 
promotion and tenure for college instructors r and therefore 
greater need to evaluate their teaching systematically. Since 
research on SETs in high schools is also scanty some of the 
following arguments necessarily draw from college-level studies. 

Though most would agree that student learning should be the 
principal output of schools^ research on student evaluations of 
teaching (SETs) shows that SETs complement student achievement 
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and other measures of school outputs with the following 
advantages: 

1 . SETS can measure dimensions of the schooling process > 
along with other dimensions of its products : Standardized! tests 
"were not designed to reveal the variety of ways in which 
teaching and learning can be creative^ favorably opportunistic ^ 
and uniquely meaningful to students.^' SETSr on the other hand^ 
can be designed to measure at least some of these many teaching 
process variables. 

2. SETS r eflect a unique and central viewpoint : Students 
see a broad range of their teacher's behaviors over months of 
exposure and personally know the students with whom they see the 
teacher interact. Furthermore^ as the intended receivers of 
teaching^ students are the only true first-hand sources regarding 
the receiving process and its effects. 

3. SETS can ref ? ect the quality of student life in school t 
which some believe is important in itself : Jencks et al. , were 
cited above making this argument persuasively, i.e., urging that 
schools be evaluated for immediate effects on teachers and 
students rather than for long-term effects on alumni (see Section 
1).62 Surely with st much of our childhood spent in school/ it 
makes sense to remove any unnecessary alienation or sadness that 
schools might produce. But schools are hardly likely to learn 
the existence, much less the source, of these ill feelings if 
students have no mechanism to reveal them. 

4. SETS can measure noncognitive traits (e.g^ , enthusiasm ) 
exhibited by teachers, and students. Recant research suggests 
that such traits are at least as important to students' later 
success as are cognitive ones: In their latest study, Who Gets 
Ahead, Jencks et al. find: 

Taken together, noncognitive measures explained at least 
as much of the variance in men's status and earnings as 
test scores did. While we could not isolate any single 
personality characteristic that was critical Co success, 
we can say that the relevant traits are largely indepen- 
dent of both cognitive skills and parental status. 

With noncognitive traits so important to later success, it makes 
sense to begin measuring them and exploring their inter- 
relationships. Indeed, SETs carefully designed to reflect 
noncognitive traits of both teachers and students, could aid 
research into the synergy between them. 

5» Tho act of seeking feedback through SETs can be a tanc;l>> 
ble ^oig n to studbr ^.s that teachers and officials seek and value 
stu der views. Moreover, the extent to which those views are 
act^ ^aaKe'^this a step, albeit a small one, toward some demo- 

Civa on of school decision making. 
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6* SETS are highly r eliable* And sinc e they are less 
correlated with student characteristics (e.g., SES) than are 
achiavemen t tests ^ they are fairer to teachers > In summarizing 
the empirical evidence on SETsr Gage writes: 

As for reliabilityr the results are almost uniformly 
happy. Averaging the ratings of about 20 students on a 
single item concerning a set of teachers j such as a rating 
of the clarity of the teacher's explanationsr yields a 
mean rating that has a reliability of about .8 or .9. 
That is, these mean ratings would correlate about .8 or 
•9 with the mean ratings of another set of 20 equivalent 
students. This finding has been obtained with almost 
perfect consistency during the 50 years or so since 
Remmers (1929) first established. In shorty the poolad 
iratings of 20 or more students reveal substantial agree- 
ment within a class as compared with the amount of 
variation between classes. 

This is consistent with "split class" reliabilities averaging .85 
for SET factor scores found in junior-senior high-school classes 
by Veldman and Peck.^^ 

Achievement tests r on the other hand, are ordinarily highly 
correlated with student characteristics. Glass writes: 

Aside from the irrelevance of much of the content of 
standardized achievement tests , their use in evaluating 
teachers is unjust. Nonrandomly constituted classes give 
teachers of brighter pupils an unfair advantage. This 
remains true whether the statistician calculates simple 
gains r residual gains , true gains , true residual gains ^ 
covariance adjustments , etc. o . . available evidence indi- 
cates that teachers* effects on pupils* knowledge are not 
reliably measured by such tests. 

Thus SETS are fairer to the teacher in the sense that they are 
less sensitive to between-student differences (or in the words of 
Gage^ they "reveal substantial agreement within a class as 
compared with the amount of variation between classes"). That 
is, they are fairer than are achievement tests ^ because they are 
less sensitive to student differences (like student's 
socioeconomic status) clearly beyond the teacher's control. 

7. SET responses correlate positively with evaluations doue 
by teacher-colleagues a nd supervisors . At the University of 
Washington, Guthrie *° found correlations ranging from 0.30 to 
0.63 between students* ratings and teacher-colleague ratings of 
the same teachers. And at the junior-senior high school level, 
Veldman and Peck^^ show agreement between SET results and 
evaluations by teacher's supervisors. This is consistent with 
the following argument: Though teachers may question the 
validity of student ratings, teachers usually know which of their 
colleagues they'd prefer their own children to study under, and 
such preferences tend to agree with the very ratings the teachers 
discount. 
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8« SETS offer wide applicability across courses : Unlike 
standardized tests ^ one SET form can apply to a wide variety of 
school subjects r thus permitting control of instrument type 
across courses. 

9. SETS may help improve instruction ; Most instructors 
would aviknowledge that certain student feedback can help improve 
instruction (especially those who^ because of feedback r no longer 
block tfee view of their own chalkboard work). And many studies'^ 
7T 72 73 74 75 show greater gains in student ratings for 
teachers who have received and acted on SET feedback (though 
rated improvement has yet to be proven equivalent to actual 
improvement) . 

10, set's are valid in that they relate positively — though 
not strongly — to student achievement ; The central challenge to 
the advocate of SET's as a measure of teaching effectiveness r is 
to disprove the likelihood of either of the two following errors: 

a. The demanding teacher produces much achievement but 
receives low student evaluations (Type I error: 
good teacher rated as poor teacher); 



b. The well-liked easy teacher produces little student 
achievement but get 
poor teacher rated 



achievement but gets high ratings (Type II error: 

:ed as excellent teacher),'^ 



The most celebrated studies to this end are the series of "Dr* 
Fox studies," These began when Naftulin, Ware, and Donnelly'' 
attempted to prove that a charismatic though nonsubstantive 
lecturer could receive high ratings even from highly educated 
professionals. Remarkably, a large number of the listeners — 
including • psychiatrists, psychologists, and educators — did 
evaluate the Dr. Fox lecture favorably; even though, according 
to the authors, the lecture contained no substantive information. 
The authors conclude that this "suggests to the educator that the 
extent to which his students are satisfied with his teaching, and 
even the degree to which they feel they have learned- reflects 
little more than their illusions of having learned."^ Yet 
without any effort whatsoever to test whether, or how much, the 
listeners may have learned, the authors conclude much more than 
their results support. 

The authors* approach improved considerably in three later 
studies,^ In short, those studies actually measured how 

student satisfaction and student achievement varied across 
different levels of teacher enthusiasm, and measurable lecture 
content. In the last of the above studies, they conclude "• • • 
the results observed to date suggest that student ratings of 
highly expressive instructors may not reflect two important 
dimensions of teaching effectiveness, namely ^ substantiveness of 
instruction and degree of student achievement."^"^ 
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FIGURE 2.1 

DR. FOX STUDIES: PROPORTION OF ACHIEVEMENT AND SATISFACTION 
STATISTICALLY ASSOCIATED V7ITH LECTURE CONTENT VER^^S TEACHER 



ENTHUSIASM AS MEASURED BY OMEGA- SQUARED' 
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ENTHUSIASM/ CONTENT RATIO 



STUDY 2 
1975 



0.09 
0.04 



2.25<- 



0.24 
0.04 



STUDY 4 
1977 



0.13 
0.06 



•>2,17 



0.24 
0.04 



6.00< >6.00 



84 



What they fail to report about their own data, however, is 
the remarkably consistent across-study pattern concerning the 
percentage of achievement and satisfaction attributable 
to lecture content versus teacher enthusiasm (as measured by 
omega-squared, the r-squared analog for ANOVA). Across both 
studies, variation in enthusiasm accounts for nearly twice (2.25 
for Study 2 versus 2.17 for Study 4) the variation in achievement 
that variation in lecture content does. And again, across both 
studies, enthusiasm accounts for exactly 6. 00 times the variation 
in satisfaction that content does. This surprising consistency 
suggests the intuitively appealing hypothesis that if lecture 
content is held constant and high, then how much of that content 
is learned, will depend strongly on how enthusiastically the 
teacher delivers it. This makes sense, since little is likely to 
be learned, regardless of how content-rich the lecture, if the 
lecturer fails to obtain the students' attention. 

Though none of the Dr. Fox material or any other research 
categorically proves or disproves the likelihood of either the 
above Type I or II errors (a fact that should caution any 
interpretation of SET results), the most carefully conducted 
studies of the correlation between SET's and student 
achievement^^ °° °' °^ do show, on average, low positive correla- 
tion between ratings of teachers' overall effectiveness and 
students' final exam score. In fact, these correlations (e.g., 
those of Sullivan and Skanes)°^ are higher than those found in 
meta-analyses seeking what teacher behaviors affect achieve- 
ment.90 M 

In meta-analysis of links between ratings and achievement^ 
Cohen used 41 independent validity studies reporting on 68 
separate multisection college courses. He found achievement 
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correlated an average of •43 with overall instructor rating ^ and 
0.47 v^ith the overall course rating. He concludes: 

We can be quite confident that the relationship between 
ratings and achievement described in this meta-analysis 
is characterized by what Bracht and Glass (1968) term 
"external validity." That is^ the present findings can 
be generalized to different students^ instructors ^ insti- 
tutions r and subject matter areas ... Based on the 
findings of the meta-analysis ^ we can safely say that 
student ratings of instruction are a valid index of 
instructional effectiveness. Students do a pretty good 
job of distinguishing among teachers on the basis of how 
much they have learned. Thus, the present study lends 
support to the use of ratings as one component in the 
evaluation of teaching effectiveness.^^ 

Indeed r the majority who have seriously studied SETs ( Cohen ^ 
Gage^ Costin et al.) conclude that they do relate, though not 
strongly r to student achievement. 

Do the teachers who are easy graders get the highest 
ratings? Some studies say that they do, others say that 

they do not^ and still others that report mixed results.^' 

Costin et al. suggest that those positive relationships that were 
found between rating and grade received r are likely to result 
from higher interest in the course triggering both higher ratings 
and higher grades^^ (i.e., not from desperate teachers buying 
higher ratings by giving higher grades). 

2.5 SET'S: SOMMARY VIEWS 

It must again be stressed that although this study focuses 
on high schools^ the relative scarcity of high school SET 
research meant heavy reliance here on college-level studies. 

In summarizing tHeir extensive review of SET research 
( which r like this review r covered some high school SET studies^ 
though mostly college-level research) ^ Costin et al. made this 
overall assessment: 

A review of empirical studies indicates that students' 
ratings can provide reliable and valid information on the 
quality of courses and instruction. Such information can 
be of use to academic departments in constructing norma- 
tive data for the evaluation of teaching and may aid the 
individual instructor in improving his teaching effec- 
tiveness.^^ 

However r with specific regard to using SET's for personnel 
decisions, McKeachie cautions: 

Student ratings of teaching are related to teacher 
effectiveness as measured by the achievement of the 
teacher's students. Nevertheless this does not mean 
that student ratings are sufficient evidence of teaching 
effectiveness. Ideally one would gather evidence from a 
number of sources ... Moreover, ratings should be 
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obtained over several courses and several semesters 
before being admitted as data in the faculty evaluation 
process. A rating of a particular course in a partic- 
ular semester may be influenced by special circumstances 
that affect its validity. Even when ratings from sev- 
eral courses are available ^ they need to be interpreted 
by peers or administrators who know something about the 
nature of the courses and students involved. 

Exactly which teacher traits affect students' overall 
satisfaction with the teacher? French-Lazovik found that at the 
college levels over 90 percent of the variation in student 
ratings of overall teacher effectiveness could be explained by 
student ratings on the following three teacher trait items: 1 ) 
Interprets abstract ideas and theories clearly; 2) Gets students 
interested in the subject matter; and 3) Has increased student's 
skills in thinking. Moreover^ the relative importance of these 
and other items in predicting overall student perceived teaching 
effectiveness maintained "... surprising consistency across a 
15-year time span on two quite different campuses with different 
student and faculty populations."'^' 

At the junior-senior high level, Veldman and Peck's 
research showed a 38-item questionnaire to embody these five 
factors: 1) friendly r cheerful , admired; 2) knowledgeable ^ 
poised; 3) interestina. preferred; 4) strict control; and 5) 
democratic procedure. Persisting over three separate 
semester's data sets (554 teachers total), this structure, like 
that found by French-Lazovik, appears steadfast with respect to 
time. (These factors also closely resemble those that emerged in 
this study's Sections 6, 7, and 8; namely "student-caring"; and 
"task-driven.") 

In short, the French-Lazovik results, the many studies 
reviewed by Costin et al.,'^^ and Cohen, and Veldman and 
Peck's junior-senior high-school level research suggest that 
student criteria for teaching effectiveness are substantive and 
fairly stable over time. 

3*1 METHODS, OVERVIEW 

The main empirical question posed here involves 
ascertaining the extent one California public high school 
efficiently produces teaching that interests students. 
That question will be answered by gathering two years of SET 
data; adding to it teacher salaries and variables that determine 
salary; and using regression models to find the extent the data 
satisfy microeconomics* specific conditions for allocative 
efficiency. The regression work involves specifying two 
regression models with salary and student satisfaction as the 
respective dependent variables and teacher traits as the inde- 
pendent ones; estimating the equations' respective coefficients; 
and finally, plugging those coefficients into the corresponding 
marginal product to price ratios — the ratios required by micro- 
economics to determine allocative efficiency. 

The original sample of 5,076 student-level questionnaires 
comes from a medium-sized public high school in the San Francisco 
Bay Area whose students are largely middle-class. 
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The 31 questionnaire items used to measure teacher traits 
essentially cover all achievement-producing traits noted by R.W. 
Heath and M.A. Nielson^ Rosenshine and Purst ^ and some of 
those tested by Gage and Hedges. Moreover ^ they cover most 
of those satisfaction-related traits revealed by the earlier- 
mentioned French-Lazovik^ ^ ' and Veldman and Peck^^^ studies. 

3.2 OBTAINING A MEASURE OF OVERALL STUDENT SATISFACTION 

Overall student satisfaction with teacher performance was 
measured as the percentage of each teacher's students who 
responded "y®s" to the question "Would you take a class with this 
teacher again?" This measure is analogous to the economists' 
notion of "repeat buyers/' i.e.^ it does the best we can do in 
practice to ascertain who would go back for more of the same 
teacher^ if free to do so. More specifically^ though economists 
would much rather observe the act of repeat buying » rather than a 
mere response to the question "would you buy again the 
rigidities of high school class scheduling generally prevent the 
student from ever performing that act. 

3.3 SINGLE-SHEET SUMMARY OF MODEL AND METHODS 
Methodologies here embody both process and input-output 

views of schooling. More specif ically, exploring links between 
schooling dollars and student satisfaction begins by articulating 
processes that should efficiently translate budgets into teaching 
that attracts r challenges r and fulfills students — a process sum- 
marized on the single sheet of Figure 3.1 and described as 
follows: 

The roughly 70 percent of district budgets that go for 
teachers' salaries can be disaggregated into components key to 
this study. Total salary (S,p, or Salary Total) appears in Figure 
3.1 as a vector — the vector sum of that component that exactly 
compensates the teacher for traits yielding desired impact 
(labeled Sj for S^alary yielding desired ^mpact) plus that resid- 
ual (Sj^r or Salary Residual) having nothing to do with that 
impact (e.g.r rewards* for remaining with the district). In 
theory r then^ that part of total salary that compensates desired 
impact is the projection of Sj onto Sip. In turn^ Sj can be 
viewed as the sum of manifold cost vectors^ each corresponding to 
a specific impact-producing teacher trait. 

The total impact a teacher has on students over time (shown 
in Figure 3.1 , as "impact vectors" labeled with I's) is then 
proportional to levels of certain teacher traits and school and 
student characteristics (e.g.^ powerful ncnteacher influences 
like student's socioeconomic status). This impact stimulates 
cognitive achievement^ perceptions of teacher traits^ and other 
outcomes (shown with arrov;s pointing different directions). 

In short r this study explores those perceptions by having 
students indicate how much their teacher exhibits specific traits 
(indicated by the PT^j^'s^ or Perceived Trait i evaluated by the 
kth student} and also whether the teaching would attract them to 
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FIGURE 3.1 
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AND THIS STUDY'S METHODOLOGIES 
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take a course with the teacher again (shown by the YNj^, a Yes or 
No response of the kth student). Teacher-level aggregates- 
Perceived Traits Averaged ^ shown by PTA ^ ' s » and the percent 
responding"^" Yes" to the above teaching question (the Y^' s) — were 
then generated from the student- level questionnaires. 

The PTAj*s served as independent variables ^ and Y^'s and 
teachers* salaries ^ as dependent variables for the marginal 
impact and marginal price regression models^ respectively (as 
shown in the lower right hand corner of Figure 3.1 ^ labeled "THE 
PRINCIPAL REGRESSION EQUATIONS"). Several different regression 
models (simple planar ^ simple quadratic ^ Cobb Douglas, logit and 
probit) helped estimate the marginal impacts of teacher traits on 
overall student satisfaction (components of the I vector: I^, 
l2r • • • Im — shown under Figure 3.1 's heading "THE IMPACT-TO- 
PRICE RATIOS FOR EACH TRAIT") and the implicit marginal prices 
paid for those traits (components of the P vector: P^, 

• • • Pm)» The simple planar equations in the lower right 
corner of the flow chart produced linear estimates of the raw 
I*s and P's, which , in turn, represented the increment in student 
satisfaction (percent total yes) and the marginal price (the 
implicit marginal increase in total teacher salary. Sip), 
respectively r per unit change in a teacher's class-average score 
for trait m, ceteris paribus. 

In short r computing the marginal-impact-to-marginal-price 
(l/P) ratio for each trait produced disparate l/P ratios across 
traits. This, in turn, suggested specific inefficiencies and 
corresponding policy implications. 

4.1 OVERVIEW OF HOW THE VARIABLES INTERRELATE: 
THE MANAGERIAL GRID FOR TEACHERS 

Perhaps the best overview of how all the variables inter- 
relate is seen in the following SPSS-X*^^ quartimax * * 
rotation in Table 4.1: 
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TABLE 4.1 

QUARTIMAX ROTATION, FACTOR MATRIX USING 
ALL TEACHER-LEVEL VARIABLES, BOTH YEARS COMBINED 

FACTOR 1 FACTOR 2 FACTOR 3 FACTOR 4 

STUDENT-CARING (THIS ST UDY) 

I PEOPLE -CENTERED (BLAKE AND MOUTON) 

UNDRST .92764 CONSIDERATION (FLEISHMAN, ET AL. ) 

INVOLV .88818 SOCIO-EMOTIONAL SPECIALIST (BALES) 

RESPEC .88715 RELATIONSHIP MOTIVATED (FIEDLER) 

FLEXBL .88491 SOCIAL EXPRESSIVITY (COUCH) 

CLEARX .86691 

WARM .85078 

YRCONT .84660 

UNBIAS .84174 

PERSON .83669 

CREATV .83627 

OPIN .80619 

COMMUN .79943 

IMPORT .78704 

PYEG .78086 

AMNTWK .76820 

FAIR .76208 

ADMIT .75304 

ENTHUS .71907 

METHOD .6 7314 

RELEV .59898 

INFORM .55147 

CONTRL .55105 



PRINTSB .51459 TASK-DRIVEN (THIS STUDY) 

I PRODUCTION-CENTERED 

PREPAR .90605 (BLAKE AND MOUTON) 

ORGNZD .87681 INITIATING STRUCTURE 

CTIME .82032 (FLEISHMAN, ET AL. ) 

KNOWL .78549 TASK SPECIALIST (BALES) 

MISSED -.63689 TASK MOTIVATED (FIEDLER) 

AVAIL .58244 TASK SERIOUSNESS (COUCH) 

RETURNS .57769 

CRITIC .51694 .54117 



* MANAGERIAL GRID FOR TEACHERS, T. Foote, 2/28/88 



PAGE 21 



TABLE 4.1, CONTINUED 
QUARTIMAX ROTATION, FACTOR MATRIX USING 
ALL TEACHER-LEVEL VARIABLES, BOTH YEARS COMBINED 



FACTOR 1 



FACTOR 2 



FACTOR 



FACTOR 4 



PHWOCCA 

PHWALWS 

HMWORK 

LGRADE 

TESTS 



-.95045 STUDENTS' 
.95045 WORK AND 

-.92664 EVALUATION 
.61684 
.51948 



PHADTO 

PROTHER 

PRTEACH 



REASON FOR 
TAKING THE 
COURSE 



-.97378 
.96245 
-.90992 



FACTOR 



FACTOR 



SALARY 

YEARS 

EDUNITS 

PBLKHIS 

PHISP 

PWHITE 



.91782 
.87422 
.80756 



SALARY 
AND ITS 
DETERMINANTS 

.95620 
.89316 
-.8241 4 



ETHNICITY 
MIX IN CLASS 



Though the questionnaire was never consciously designed to 
produce specific factors, the above loadings suggest variable 
groupings with the follov: descriptions: 

FACTOR 1 : The extent to which the teacher cares about the 
socio-emotional well-being of students (STUDENT- 
CARING). 

FACTOR 2: The degree to which the teacher is driven toward 
completion of the tasks at hand (TASK-DRIVEN); 

FACTOR 3: The student's work and evaluation; 

FACTOR 4: The student's reasons for taking the course; 

FACTOR 5: The teacher's salary variables; and, 

FACTOR 6: The ethnicity mix of the students within a given 
teacher's aggregation; 

The first two factors, STUDENT-CARING and TASK-DRIVEN, 
account for 69.6 percent of the total variation among teacher 
trait variables alone ' ' ' and replicate the same two latent 
dimensions that persist across other studies from psychology and 
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FIGURE 4.1 
THE MANAGERIAL GRID OF BLAKE AND MOOTON 
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managerial leadership (e.g. , Bales, Blake and Moutcn,^ 
Couch, Fiedler, and Fleishman et al.**'). Labeling these 
dimensions as "concern for people" and "concern for production" 
Blake and Mouton plot and discuss five managerial styles on their 
resulting "Managerial Grid," reproduced in Figure 4.1. According 
to Blake and Mouton, that grid: 

... has been applied in widely different organizational 
settings in the United States, Canada, Europe and Asia. 
Included are industrial facilities of manufacturing, 
sales, R&D, and union organizations as well as military, 
governmental, professional and welfare settings such as 
community agencies ... it seems to provide descriptions 
of managerial alternatives that are equally useful when 
applied to managerial dilemmas in the U.S., in countries 
of Europe that are somewhat similar to our own, and in 
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cultures of Asia^ which are far different* The schema ^ in 
other words r seems to be relatively culture-free and^ 
therefore r of general relevance for understanding problems 
of management wherever men work in concert • • • its 
application is not limited to any particular level in the 
organization hierarchy. It applies in solving problems at 
the bottom where concrete supervisory skills are required 
and at the top where executive decision-making involving 
far more subtle and complex judgments is demanded.^ '° 

Likewise r Fleishman ^ Harris and Burtt describe what appear to be 
the same factors discussed by Blake and Mouton: 

Actually^ we were able to extract two leadership factors 
which were quite independent^ both of which will be dis- 
cussed in detail later. One identified as 'consideration' 
included such characteristics between supervisors and 
subordinates as friendliness, mutual trust, and respect. 
The other r termed 'initiating structure involved estab- 
lishing patterns of organization, channels of communica- 
tion ^ and ways of getting things done. 

The four teacher traits loading heaviest on the "student 
caring" versus "task driven" factors of this study's "Managerial 
Grid for Teachers r" and the other authors' respective versions of 
its dimensions r appear in Table 4.2. 
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TABLE 4.2 

POUR VARIABLES LOADING HEAVIEST ON STODENT-CARING AND TASK-r»IVEN FACK»S 
AND OTHER AUTHOR'S LABELS FOR IHEIR VEEISIONS OF THESE FACTORS 



FACK» LABELS 
FOR THIS STUDY 



STUDENT-CARING 

DEIGREE TO WHICH YOUR TEACHER: 



TASK rnimn 

DEGREE TO WHICH YOUR TEACHER: 

1 . IS PREPARED FOR CLASS 
(PREPAR) 

2. IS WELL C«GANIZED (ORGNZD) 

3. MAKES CONSTRUCTIVE USE OF 
CLASS TI^E (CTIME) 

4. IS KNOWLEDGEABLE ABOUT THE 
SUBJECT MATTER (KN^JL) 



VARIABLES 
LOADING 
HEAVIEST ON 
THESE FACTORS 



1. 
2. 



3. 



4. 



IS UNDERSTANDING (UNERST) 

STIMULATES YOUR INTEREST 

AND INVOLVEMENT IN THE 

COURSE (INVOLV) 

IS RESPECTED BY YOU 

(RESPEC) 

IS FLEXIBLE (FLEXBL) 



AUTHORS' LABELS FOR SIMILAR FACK^ FOUND IN PSYCHOLOGY AND 
MANAGEMENT LITERATURE: 



BALES 

BLAKE AND 
MOUTON 

COUCH 

FIEDLER 

FLEISHMAN, 
ET AL. 



S0aO-EM0TI<mL SPECIALIST 



EEOPLE-CEinERED 



SOOAL EXPRESSIVITY 



RELATIONSHIP MOTIVATED 



CONSIDERATION 



TASK SPECIALIST 



PRCXXJCnai-CENTERED 



TASK SERIOUSNESS 



TASK MOTIVATED 



INITIATING STRUCTURE 



The other authors' labels for their versions of these two 
dimensions have also been inserted in the following plot of the 
factor loadings (Figure 4,3), 

It should be no surprise that evaluations of teachers 
produce the very two dimensions widely believed to be key among 
managers. Though big differences exist between their respective 
jobs, both are responsible for getting a subordinace group of 
people to accomplish prespecified tasks, and reward them for 
it. 
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FIGURE 4.3 

QUARTIMAX ROTATION PLOT OF FIRST T170 OF THE ABOVE FACTORS: 
A MAHAGERIAL GRID FOR TEA. ;^ERS 
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DEIEV.4 STUD.-ORING 
OCEglELATICN. 



AVAIL4 9CRITIC 
ASSIGM3 
EIETIWI8 
CDNIRaL? 



SALARY PGR MAX 
SIUCQIT CARIt^ 
CORRELASJCN 



EixiNrrsii 

MASIERS22 



TESTS47 



ACMITI 



NSTUDS25 



24641 39PRC7IHER 
I28EHISP 



niMPCRT 

248UNBIAS 
I 44RESFEC 
A^NIWK 
52YRaDNT 



t 



YEARS EXFEEUSO: 

'■ i S-T-U-D-E-N-T C-A-R-I-N-G >♦ 

5129PBLKHIS 21 RE[£7432312 6 49UNDERST 
LGRAEE 5019INVOLV 
PERCENT YES >42 8CREAT7 



EERICD30 

PCfniER3633PHAETO 
ERIEACH4035PHW0OCA 



MISSED24 



THE MKE TASK-OmrEN THE TEACHER 
TEE ETMER THE TIMES OHAT STODENTS 
MISSED CLASS. 



14FLEXBL 
260PIN 



38ERINTSB 
15GRA0E 



STODENT-CARING ('IHIS STUDY) 
EBOPr£-CEimKED (BLAKE AND MXTTON) 
O0NSIDE31ATI0N (H.EISHMAN, ET AL.) 
SCaO-EMDnONAL SPECIALIST (BALES) 
RELATICNSHIP MOTIVATED (FIEDI£R) 
SOCIAL EXPRESSIVITY (OCUCH) 



Note: Tlie overall satisfaction measure, PERCENT YES, loads on TASK- 
ERIVEN negatively here, but positively in other rotation methods. For the 
main analyses, however, the vcirible <±osen to represent TASK l3?IVEi4, "IS 
PREPARED FOR CLASS" (variable 37 above) does relate positively to satisfac- 
Uon. 
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4.2 THE GRID AND RELATIONSHIPS AMONG KEY VARIABLES 

The meanings of other variables plotted in Figure 4.3 also 
seem to agree with their locations within the TASK-DRIVEN versus 
STUDENT-CARING grid. For example r the strongly negative loading 
of "how mai/y class periods of this course have you missed" 
(MISSED24 on the plot) confirms that the more task-driven the 
teacher^ the fewer the class periods the students will miss (or 
will report having missed). 

Of considerably greater importance r here^ is that overall 
satisfaction (the percentage of students who want to take a . 

course again with the teacher^ PERCENT YES >42 on the plot) 

loads quite positively on STUDENT-CARING r and^ ambiguously on 
TASK-DRIVEN (though its positive correlation with the strongest 
proxy for task-driven^ is less unambiguous) Moreover , its 
loadings are distant from^ and carry opposite signs of, the 
loadings for salary. Taken together, this suggests that student 
satisfaction relates strongly to student-caring, considerably 
less to task-driven.7 and perhaps negatively, if at all, to 
salary. In short, this hints that salary is not allocated to 
produce satisfaction efficiently. 

5. SPECIFYING^ AND ESTIMATING THE EQUATIONS 

Despite distinct differences in each rotation method's 
objectives. Table 5.1 shows that for this particular sample (both 
years of teacher- level aggregates combined and weighted) each 
produced, roughly similar rank-orderings of variables loading high 
on each of both factors. Indeed, the consistently high loadings 
of the same variables across the five different techniques argues 
for using these variables as proxies for the grid dimensions they 
load high on. Substituting proxies for the grid dimensions will, 
in turn, avoid what in earlier runs involved messy computations 
and interpretations of factor scores; a substitution consistent 
with our goal of constructing a model that is parsimonious, and 
easy to interpret. 

Table 5.1 shows that the STUDENT-CARING versus TASK-DRIVEN 
factors will be best represented by either UNDERSTANDING and 
ORGANIZED (ranked first on each of the two dimensions for 
quartimax) or UNDERSTANDING and PREPARED (first ranked for both 
varimax and equamax). 
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TABLE 5.1 

RANK ORDERING OF FOUR HIGHEST LOADING VARIABLES ON 
STUDENT-CARING VERSUS TASK-DRIVEN FACTORS 
FOR DIFFERENT ROTATION TECHNIQUES 



QqARTIMAX 



VARIMAX 



B3UAMAX 



OBLIMIN 



STUDENT-CARING: 



FIRST 
SECOND 
THIRD 
POURIH 



UNDERSTANDING ^UNDERSTANDING UNDERSTANDINO^FT.EXTRT.E 

RESPECTED FLEXIBLE ^FLEXIBLE — """'"^UNDERSTANDING 

PERSOi ^YOUROCKERIB YOUROCMRIB YOURCONITRIB 

YOUROCMRIB WARM WARM ^WARM 



TASK-ESIIVEN: 



FIRST 
SECCMD 
THIRD 
PQURIH 



CRGANIZED...,^^^PREPARED ^PREPARED --^^^Q^uANIZED 

PREPARED --^^^^ ORGANIZED CRGANIZED-'^^^^PREPARED 

CLASSTIME CLASSTIME CLASSTIME CLASSTIME 

KNOWLEDGE KNOWLEDGE KNOWLEDGE KNOWLEDGE 



Where r in alphabetical order r the above variable names correspond to the 
following questionnaire items ^ measuring the extent to vMch the teacher 
exhibits the indicated tra5.t: 



CLASSTBE: 
FLEXIBLE: 
INVOLVED: 
KNCfrJLEDGE: 

orgainIIzed: 

PERSON: 

PREPARED: 
RESPECTED: 
UNDERSTANDING: 
WARM: 

YOUROONTRIB: 



Makes constructive use of class time. 
Is flexible • 

Stimulates your interest and involvenei in the course. 
Is knowledgeable about the subject matter. 
Is well organized. 

Respects you as an individual; seens to be interested in you as 
a person. 

Is prepared for class. 
Is respected by you. 
Is understanding. 
Is warm/friendly. 

Makes you feel that your contributions are important and 
meaningful. 



5.1 ESTIMATING THE COEFFICIENTS 

After numerous attempts with more complicated models r the 
preferred models contain only the earlier determined proxies for 
each of the grid dimensions ^ and the inputs the district actually 
pays for (teacher's post-B.A. education units ^ and teacher's 
years experience) ^ for a total of four independent variables. 

Regressions were done U3ing five models: simple planar^ 
simple quadratic, Cobb-Douglas, probit, and logit. In short, the 
simple planar model has PYES move as a linearly increasing 
function of a given trait variable (ceteris paribus), while the 
quadratic has PYES move as a curvilinearly increasing (squared) 
function — with decreasing sl'^pe (diminishing returns). Though 
Cobb-Douglas, probit and logx are also curvilinear, Cobb-Douglas 
embodies a slightly more sophisticated exponential of which the 
natural log is taken; Probit involves the cumulative normal 
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TABLE 5.2 

ADJUSTED R- SQUARED VALUES FOR THE DIFFERENT MODELS 
RUN USING SPSS-X'S WEIGHTING METHOD 



STUDENT CARING PROXY > UNDERSTANDING 

TASK-DRIVEN PROXY >ORGANIZED 



UNDERSTANDING 
PREPARED 



1. SIMPLE PLANAR 

A. DIRECT 

B. CONTROLLED 



0.536 
0.408 



(RUN 3) 
(RUN 4) 



0.536 
0.409 



(RUN 
(RUN 



11 ) 
12) 



2. SIMPLE QUADRATIC 

A. DIRECT 

B. CONTROLLED 



0.609 (RUN 5) 
0.490 (RUN 6) 



0.627 (RUN 13) 
0.506 (RUN 14) 



3. COBB-DOUGLAS 

A. DIRECT 

B. CONTROLLED 



0.346 
0.406 



(RUN 7) 
(RUN 8) 



0.331 
0.371 



(RUN 
(RUN 



15) 
16) 



4 . PROBIT 

A. DIRECT 

B. CONTROLLED 



0.422 (RUN 9) 
NOT RUN 



0.417 (RUN 17) 
NOT RUN 



5. LOGIT 

A. DIRECT 

B. CONTROLLED 



0.479 (RUN 10) 
NOT RUN 



0.475 (RUN 18) 
NOT RUN 



6. SIMPLE PLANAR REGRESSION INCLUDING THE ACTUAL PAID-FOR 
INPUTS, TEACHERS POST-B.A. UNITS, AND TEACHER'S YEARS 
EXPERIENCE. 

STUDENT CARING PROXY J UNDERSTANDING FLEXIBLE 

TASK-DRIVEN PROXY > ORGANIZED PREPARED 



A. DIRECT 

B. CONTROLLED 

C. SALARY EQUATIONS 



0.570 (RUN 1 ) 
0.451 (RUN 3) 
0.951 (RUN 5) 



0.570 (RUN 2) 
0.453 (RUN 4) 
0.954 (RUN 6) 



probability function; And logit uses a "logistic function" of the 
odds that a given student will respond "yes" to the question 
"Would you take a class again with this teacher?" 

Table 5.2's rows labeled "DIRECT" indicate where PYES was 
used directly as the dependent student satisfaction variable, 
while rows labeled "CONTROLLED," refer to regressions that con- 
trolled PYES for effects beyond the teacher's control (effects 
discovered earlier, which we had promised to deal with at this 
regression stage). 

Despite the virtues of "controlled" regressions over 
"direct" ones, direct versions were still run to understand the 
nature of the raw production function, and to see the degree to 
which the controlling process "dulls" the regression equations in 
terms of lower R-squareds, wider standard errors about the 
coefficients, and lower significance levels. As the adjusted R- 
squareds of Table 5.2 show, this dulling effect is considerable. 
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5.3 SIMPLE PLANAR APPROACH 

Consistent with the hope that the model be parsimonious, the 
earlier-mentioned trait proxies for STUDENT CARING (UNDRST) AND 
TASK DRIVEN (PREPAR) were plugged into the following planar 
model : 

PYES = Bq + (STUDENT CARING) + B2 (TASK DRIVEN) 
Table 5.3 shows the resulting coefficients: 



TABLE 5,3 

COEFFICIENT RESULTS: SIMPLE AND RESIDUAL PLANAR MODELS 



RUN 11 SIMPLE PLANAR REGRESSION OF: 
ADJUSTED R-SQUARED = 0,536 



INDEP, VARIABLE 



PYES = F(UNDRST, PREPAR ) 



SLOPE B STD ERR B 



T SIG T 



PREPAR 
UNDRST 
(Y- INTERCEPT) 



5,22761 
31,08833 
-111 ,58839 



2,01345 
1 ,45772 
11 ,29216 



2,596 
21,327 
-9,882 



,0097 
,0000 
,0000 



Though the planar fits offer respectable R-squareds, the 
linearity embodied in the above will not reveal the extent to 
which diminishing marginal returns (curvilinear phenomena) might 
operate, 

5.4 SIMPLE CURVILINEAR (QUADRATIC) APPROACH 

To detect and measure marginal returns, the following 
quadratic form was run on both pairs of factor proxies: 

PYES = Bq + B^ (STUDENT CARING) + B, (STUDENT CARING )2 
+ B3 (TASK DRIVEN) + B4 (TASK DRIVEN )2 

As Table 5,2 reveals, this particular model explained more varia- 
tion in satisfaction than did any other, in both direct, and 
controlled versions. Moreover, as Table 5,4 shows, it consis- 
tently yielded negative coefficients on the squared terms — i,e^, 
empirical evidence the diminishing returns discussed above. 
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TABLE 5.4 
COEFFICIENT RESULTS 
FOR THE SIMPLE QUADRATIC MODEL 



RUN 13 SIMPLE QUADRATIC REGRESSION OF: 

PYES = F(UNDRST, PREPAR, UNDRST"^, PREPAR "^ ) 

ADJUSTED R-SQUARED = 0.627 



INDEP. VARIABLE 


SLOPE B 


STD ERR B 


T 


SIG T 


(Y-INTERCEPT) 


-1619.28693 


176.32957 


-9.183 


.0000 


UNDRST 


280.19966 


29.58726 


9.470 


.0000 


PREPAR 


327.74262 


64.15070 


5.109 


.0000 


UNDRST^ 


-24.53956 


2.91113 


-8.430 


.0000 


PREPAR^ 


-29.53668 


5.91444 


-4.994 


.0000 



5.5 CLOSER INSPECTION OF PLANAR AND QUADRATIC FORMS 

Since both the planar and quadratic forms require no trans- 
lation of any of the original variables (unlike Cobb-Douglas, 
Idgit, and probit), both can be discussed in the context of 
three-dimensional plots whose dimensions measure the locations of 
teacher aggregates in the relevant production space* That space 
will measure student satisfaction (PERCENT YES) as a function of 
teaching that is student-caring (proxied by UNDERSTANDING), and 
task-driven (proxied by PREPARED).'^* The 60 actual teacher 
aggregates plotted^ in that space appear in Figure 5*2: 
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FIGURE 5.2 

ACTUAL, PERCENT YES AS A FUNCTION 01'' 
. UNDERSTANDING VERSUS PREPARED 
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As the above plot confirms ^ the high ratings received on 
average by all teachers ^ confine the data to the greater-than-4,0 
quadrant of the original floor. All 60 data points do not appear 
in the plot because the plotting software ^ designed to show the 
corresponding surface ^ depicts only the approximate view that 
would result from draping a sheet over the points. That is^ it 
conceals all data points that lie under the sheet (All the actual 
data points are more conveniently plotted and discernible in 
subsequent figures depicting two-dirtiensional slices of the pro- 
duction space). Despite the concealment ^ the surface does sug- 
gest that PERCENT YES tends to rise as a function of the 
teacher's average for PREPAREDr and UNDERSTANDING, Plots of the 
corresponding planar and quadratic fits (whose statistics were 
detailed earlier) confirm the nature of that trend (Figures 5,3 
and 5, 4), The four- term quadratic equation produces the expected 
surface with positive firsts and negative second, derivatives 
with respect to both inputs. 
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PERCENT YES AS PLANAR FUNCTION OF 
UNDERSTANDING VERSUS PREPARED 




ACTUAL SAMPLE SIZE WAS N = 60 TEACHER AGGREGATES OF 3.961 STUDENTS. 
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PERCENT YES AS A QUADRATIC FUNCTION OF 
UNDERSTANDING VERSUS PREPARED 
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Detailed comparison of the planar versus quadratic fits is 
enhanced by taking vertical slices of the surfaces^ at the means 
of the respective inputs • For example, Figure 5.5 represents an 
enhanced version of the slice shown in the three dimensional 
quadratic plot of Figure 5, 4, This PERCENT YES versus PREPARED 
cross section taken at the average value for UNDERSTANDING (equal 
to 5.17) shows both the corresponding planar and quadratic 
curves t and superimposes projections of all the actual data 
points (represented with "A"'s for Actuals or "*"'s for where two 
or more points overlap). The word projections is emphasized 
because if one misperceives the points and ct^rves to share the 
same plane, then one understates the goodness of fit between the 
data cloud and the fitted surfaces. 

Figure 5.6 shows a second slice, this time taken at the mean 
of PREPARED (equal to 5«48)« Together, Figures 5. 5 and 5. 6 
reveal, as do the equations in the bottom of each plot, that 
student satisfaction increases most steeply with respect to stu- 
dent-caring, and very gradually with respect to task-driven. 
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FIGURE 5.5 

SLICE DEPICTINS ACTUAL, PLANAR, AND QUADRATIC FITS PROJECTED AGAINST PYES VERSUS PREPAR PLANE, 
HITH UNDRST HELD CONSTANT AT ITS HEAN. SEE EQUATION DETAIL IN LOHER PORTION OF BOX. 
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A = ACTUAL PERCENTAGE OF TEACHER'S STUDENTS RESPONDING YES TO 

THE QUESTION 'MOULD YOU TAKE A COURSE HITH THIS TEACHER AGAIN?' 
P « PLANi^R FIT = PLYESPU=-U1.5G839 * 31.08833*5.17873 * 5.22761»PREPAR 
Q = QUADRATIC FIT = QYESPU=-1619.28693 * 280. 19966i5. 17873 + -24.5395ot5.I7873"2 

t 327.74i62tPREPAR + -29.5366BtPREPARH2 

f = HHERE AN A, P, OR Q, COINCIDE. 



4.86 4.98 5.10 5.22 5.34 5.46 5.53 5.70 5.82 5.94 
4.80 4.92 5.04 5.16 5.2? 5.40 5.52 5.44 5.76 .5.33 



X = TASK-DRIVEN PROXY = PREPAR = TEACHER'S AVERAGE RATING ON: 'THE 
DEGREE TO HKICH YOU THINK YOUR TEACHER IS PREPARED FOR CLASS.' 
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FIGURE 5.6 

SLICE DEPICTING flCTUAl, PLANAR, AND QUADRATIC FITS PROJECTED A6AIHST PYES VERSUS UNDRST PLANE, 
WTH PREPAR HELD CONSTANT AT ITS JIEAN. SEE EBUATION DETAIL IN LOKHH PORTION OF BOX. 
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ft » ACTUAL PERCENTAGE OF TEACHER'S STUDENTS RESPONDINS YES TO 

THE QUESTION 'KOULD YOU TAKE A COURSE NITH THIS TEACHER AGAIN?' 
P « PLANAR FIT » PLYESUP"-! 11.58839 ♦ 31.08B33IUNDRST ♦ 5. 22761 ♦5. 47806 
0 « QUADRATIC FIT » QYESUP»-1619.29693 ♦ 280. l9966iUNDRST ♦ -24.53954»UNDRST«2 

♦ 327.7426215.47806 ♦ -29.53668*5.47806**2 

* « IWERE AN A, P, OR Q, COINCIDE. 



4.275 4.425 4.575 4.725 4.875 5.025 5.175 5.325 5.475 5.S25 
4.350 4.500 4.650 4.800 4.9S0 5.100 5.250 5.400 5.550 5.700 

X » STUDENT-CARINS PROXY UNORST =» TEACHER'S AVERAGE RATING OH: 
•DEBREE TO HHICH YOU THINK YOUR TEACHER IS UNDEHSTAHOING' 
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Prior to the above plots, the quadratic model appeared to 
offer several substantial advantages over the planar form. It 
captured the most variation of all the models while producing 
significant coefficients that reflect diminishing returns and 
allow for determination of point marginal products. Yet as the 
slices show, the point marginal products (slopes) are sensitive 
to the particular grid coordinates at which those surface points 
are evaluated. By contrast, the planar fit reflects a good 
approximation to the quadratic one, and embodies what might be 
seen as an "overall average" slope that is constant and positive 
throughout the range of the respective input. Though the quad- 
ratic's point-specific slopes are more accurate than ths planar 
form's "average" view, there is no stringent need here for such 
accuracy. The planar form offers more simplicity, and, slope 
values generalizable to the entire range of input values — while 
closely approximating the corresponding quadratic slopes. More- 
over, the simpler planar model will greatly aid the final process 
that must include estimating marginal products and marginal 
prices of not only UNDERSTANDING and PREPARED, but also the in- 
puts for which the district pays explicit prices (years exper-^ 
ience and educational units). Finally the reader is reminded 
that this is the most common form of educational production func- 
tion. For the above reasons, the final regressions embodied the 
following simple planar form: 

PYES = BASE + MI^(TT^) + Ml2(TT2) + . . . + MIj^(TTj^) 



5.6 FINAL REGRESSIONS 

Table 5.3 presented coefficients for the planar form losing 
just the two grid-dimension proxies as predictors. Heweve:;, to 
analyze the school district's actual allocation policy relative 
to those two theory-based inputs, the inputs to which the 
district actually allocates (teacher's years experience, coded 
"YEAKS," and education beyond the bachelor's degree, coded 
"EDUNITS") must also be included. 

To estimate the marginal impact coefficients, YEARS and 
EDUNITS were added to the equation predicting PYES. The 
resulting coefficients appear in Table 5.5: 



TABLE 5.5 

SYSTAT WEIGHTED MODEL WITHOUT INTERCEPT 
DEPENDENT VARIABLE = PYES 



INDEP. VARIABLE 



SLOPE B 



STD ERR B 



T 



SIG T 



UNDRST 
PREPAR 
YEARS 
EDUNITS 



33.952 
5.01 5 

-1 .256 
0.087 



3.905 
5.376 
0.633 
0.104 



-4.02 
0.93 

-1 .99 
0.83 



.000 
.350 
.050 
.400 
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5*7 FINAL RATIOS 

With the marginal impact estimates in hand^ calculating the 
follov;ing marginal-impact-to-marginal-price ratios t 

MX^ ? MI2 ? ? MIj^ 

MP.^ MP2 * MPj^ 

requires that we now estimate the implicit marginal prices (the 
MP^'s of the denominators) • These are obtained by replacing only 
PYES with SALARY in the same model that generated the MI^'s, then 
rerunning the regressions; i^e.^ by changing: 

PYES = BASE + MI^(TT^) + MIjCTTj) + • • . + MIj^(TTj^) 
PYES 

to: 

SALARY = BASE + MP^ (TT- ) + MPjCTTj) + . . . + MPj^(TTj^) 
SAL 

and estimating the new coefficients. 

The coefficients from the student-weighted regressions 
predicting satisf action, combined with those from the unweighted 
regressions predicting salary , yield the long-sought "marginal 
impact to marginal price" ratios of Table 5.6^ 

The extreme disparities among Table 5.6's quotients, suggest 
that the district allocates its budget inefficiently with respect 
to producing teaching that attracts students. The quotients 
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TABLE 5 • 6 

FINAL MARGINAL IMPACT TO 'MARGINAL PRICE RATIOS 
AND RELATED STATISTICAL DETAIL 
(* => p <. .05) 



UNDRST= PREPAR= 

IS IS 

PREDICTORS > UNDER- PREPARED 

STANDING FOR CLASS 



(STD. ERROR B) 
(T STATISTIC) 
(SIG. OF COEFS. ) 
(PART. CORR.) 



( 3.905) 
( -4.02 ) 
(.000 ) 
(.73 ) 



( 5.376) 
( 0.93 ) 
(.355 ) 
(.30 ) 



YEARS= 
YEARS 
TEACHING 
EXPERIENCE 

( 0.633) 
( -1 .99 ) 
(.052 ) 
(-.19 ) 



UNITS= 

EDUCATION 

UNITS 

BEYOND B.A, 

(0.104) 
(0.83 ) 
(.408 ) 
(-.17 ) 



SLOPES FROM SYSTAT WEIGHTED NO-CONSTANT 
REGRESSION ADJUSTED MULTIPLE R-SQUARED = .430 



ACTUAL 
COEFFICIENTS 
OF PYES 

EQUATION >*33.952* 5.01 5 -1 .256 0.087 

^ ^ ^ 

ACTUAL > -169.057 *819.285* *811.026* *25.347* 

COEFFICIENTS 

OF SALARY SLOPES FROM 60 TEACHER UNWEIGHTED SALARY 



EQUATION 

(STD. ERROR B) 
(T STATISTIC) 
(SIG. OF COEFS. ) 
(PART. CORR.) 



REGRESSION ADJUSTED MULTIPLE R-SQUARED = .960 



(300.664) 
( -.562) 
(.5000 ) 
(.075 ) 



(50.357) 
(1 6.269) 
(.0001 ) 
(.909 ) 



(432.621 ) 
( 1.874) 
(.0500 ) 
(.245 ) 



(8.242) 
(3.075) 
( .0030) 
(.383 ) 



MI/MP QU0TIENTS~> -0.201 ^ 0.006 ^ -0.002 



0.003 



TABLE 7.7 

CORRELATION MATRIX OF MAIN ANALYSIS VARIABLES 
USED TO OBTAIN THE ABOVE RATIOS 



The following was computed from teacher-level aggregates 
where each aggregate was weighted by the square root of the 
number of students in the aggregate (* => p £ .05). 





EDUNITS PREPAR 


PYES 


SAL 


UNDRST 


YEARS 


EDUNITS 


1.00 .21 


-.17 


.86* 


-.10 


.82* 


PREPAR 


1 .00 


.30* 


.19 


.31* 


.12 


PYES 




1 .00 


-.20 


.73* 


-.19 


SAL 






1 .00 


-.04 


.97* 


UNDRST 








1 .00 


-.03 


YEARS 










1 .00 



themselves^ hov;ever, cloud the more useful picture offered by 
their marginal-impact and marginal-product numerators and 
denominators. That picture begins with TABLE 5.8 *s discussion of 
the coefficients: 
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TABLE 5.8 
DISCUSSION OF THE PRECEDING 
MARGINAIr-IMPACT-TO-MARGINAL-PRICE COEFFICIENTS 
(* => p <. .05) 



UNDRST PREPAR 

r r 
? 



MI 



1 



MP. 



1 



The proxy for student-car- 
ing, the degree to which the 
teacher "is understanding," 
has considerably ^rore 
impact on satisfaction than 
"is prepared" does. 



Ml2 ? 
MPo 



YEARS 



MI: 



EIXJNITS 



MI/ 



MP, 



MP. 



PYES 

EQUATION 



The proxy for task-driven teaching, 
the degree to which the teacher 
is prepared for class, is not 
statistically significant, yet it 
carries a high and statistically 
significant implicit price of $819.28. 

I ! 

If YEARS and UNITS relate 
at all to satisfaction, 
they may relate negatively. 



>y^33.952* S-Oisl 



SALARY — 
BQUAnON 



-> -169.057 



*81 9.285* 



-1.256 



1*811.026*! 



0.087 



*25.347* 



Although the price 
coefficient for UNDER- 
STANDING is not sta- 
tistia\lly significant, 
UNDERSTANDING t^s the 
most impact, of all 
variables, on student 
satisfaction. 



Explicit allocations to YEARS and 
UNITS relate tightly to salary 
because they determine salary. Yet 
they purchase no, if not negative, 
amounts of student satisfaction. 



Table 5.8 's data and comments are hardly definitive. 
Indeed, they embody many of the same limitations of the other 
research of educational production functions. Nonetheless, like 
that research, the above empirical findings do quantify and 
support much of the theory about economic and human behavioral 
phenomena in schools. 
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6.1 CONCLUSIONS 

Evidence of nationwide student dissatisfaction with school 
prompted the research questions posed in Section 1 • Those 
questions r and this study's answers to them follow: 

Question 1 : What advantages might student evaluations of 
teachers (SETs) offer California public high schools as an 
added measure of schooling effectiveness? 

Section 2 reviewed studies from colleges and high schools — 
studies suggesting that SETs could augment achievement and other 
measures of schooling outputs by offering these advantages: 

1 • SETS measure affective dimensions of the schooling 

process (i.e,^ students' and teachers' feelings about 
the process) and other dimensions of its products 
(e.g., the learner's self esteem); 

2. SETS reflect the unique viewpoint of the intended 
receivers of teaching — the most important first-hand 
sources regarding the receiving process and its 
effects- -the only sources who experience so broad a 
range of a teacher's behaviors over months of expo- 
sure; 

3. SETS reveal the quality of student life in school — 
what Jencks et al, urge we begin seeing "as an end in 
itself r rather than a means to some other end"; 

4. SETS can measure noncognitive traits (e.g,^ enthusiasm) 
exhibited by teachers , and students. In Who Gets Ahead ^ 
Jencks et al, argue that such traits are at least as impor- 
tant to students' later success as are cognitive ones; 

5. The act of seeking feedback through SETs can be a tan- 
gible sign to students that teachers and officials 
seek and value students' views, 

6. SETS are highly reliable o And they are fairer to 
teachers in the sense that they are less correlated 
with student characteristics (e.g,^ SSS) than are 
achievement tests, 

1. SET responses correlate positively with evaluations 
done by teacher-colleagues and supervisors. 

8, Unlike achievement tests ^ SETs apply to a wide variety 
of school subjects in a single instrument and thus 
permit control of instrument type across courses; 

9. SETS provide teachers with feedback which ^ evidence 
shows ^ can help them improve their teaching; and fin- 
ally, 
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10* SETS offer validity in the sense that higher student 
evaluations of teaching do correlate — though not 
strongly — with higher student achievement. This is 
consistent with findings in the "Dr. Fox" studies that 
show ratings to effectively measure teacher •s enthus- 
iasm, and enthusiasm to have twice the impact on learn- 
ing that lecture content does. 

Question 2: What fundamental dimensions of teaching under- 
lie such evaluations? 

The factor analysis reported in Section 4 showed two factors 
to predominate: STUDENT-CARING and TASK-d:;IVEN. These two 
accounted for 70 percent of the variation among all teacher trait 
variables, and replicated the same tv70 latent dimensions proven 
useful in studies from psychology to managerial leadership (e.g.. 
Bales. Blake and Mouton, Couch, Fiedler, and Fleishman et 
al.)* Borrowing from Blake and Mouton*s extensive works 

articulating management styles on the "Managerial Grid," these 
dimensions now found in teaching, provide conceptual framework 
for a "Managerial Grid for Teachers." 

Here, the grid first helped reveal — from where variables 
fell on the factor plot — interrelationships of all SET variables 
at a glance. For example, it showed that overall student 
satisfaction strongly related to STUDENT-CARING, much less 
strongly to TASK-DRIVEN, and negatively, if at all, to teacher's 
salary. In short, it told a preliminary graphic story of alio- 
cative inefficiency with respect to producing satisfaction — one 
of several stories told by several groupings of plotted varia- 
bles. 

Later, when we changed the dimensions from meaning "degree 
of correlation with a factor" to meaning "degree teacher is task- 
driven or student-caring," the grid prompted useful images and 
questions linking teaching styles to achievement, satisfaction, 
and any other variables plottable on the grid — images and 
questions discussed in this chapter's final section, 
"Implications for Future Research," 

Because other authors (e.g., Blake and Mouton et al.)'^^ 
have devoted many useful books to grid-related leadership styles 
Jor other professionals (e.g. managers, nurses, salesmen, aca- 
demic administrators) — surely the same could be done for teach- 
ers. Moreover, as will be discussed in the final section, 
augmenting the grid with achievement, satisfaction, or other 
output surfaces, provides a conceptual framework for seeing 
teaching style tradeoffs and their output consequences — 
suggesting still more questions for future research. 

In helping us seek allocative efficiency with respect to 
satisfaction alone, the grid provided the foundation ^or tackling 
the remaining research questions — those questions marking this 
study •s destination: 

Question 3: To what extent does one public high school in 
California already satisfy the microeconomic conditions to 
efficiently produce teaching that interests students? That 
is, how much impact do specific teacher traits {e.g., know- 
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ledge of the subject matter, preparation for class, enthus- 
iasm) have on overall student satisfaction? What implicit 
monetary prices do districts pay for those traits? To what 
extent are the impact-to-price ratios disparate across those 
traits? 

Section 5 examined this question by fitting planar^ quad- 
ratic r Cobb-Douglas r probit and logit models to proxies for the 
grid dimensions. After adding teacher's salary determinants to 
the model and controlling statistically for influences beyond the 
teacher's control^ coefficients were computed to produce this 
study's final quantitative product ^ the marginal-product-to- 
marginal-price ratios of Tables 5.6 and 5.8. 

The quotients' substantially unequal numerators and 
denominators — four of which are statistically significant — 
suggest failure in meeting the above efficiency conditions. More 
precisely r they imply that this particular public high school is 
not already allocating its teachers' salaries to efficiently 
produce teaching that students would like to return to. Details 
of the inefficiencies follow. 

Question 4: To what specific inefficiencies do any dispari- 
ties point and what policy changes do they imply? 

The abovo inefficiency stems mostly from the following 
linkages: 

1. Student-caring has far more influence on our satis- 
faction measure than task-driven does; but 

2. Teachers are implicitly paid far less for student- 
caring behaviors than they are for task-driven ones. 

3. Teacher's salary is driven predominantly by teaching 
experience^ and relatively less by education. 

4. Yet neither experience nor e'ucation have any appre- 
ciable impact on student satisfaction. 

In shorty the district's heavy payments to experience and 
education do not, coincidentally buy the inputs that would boost 
student satisfaction (though at $819 per rating point of "Is 
prepared for class/* they do coincidentally buy task-driven 
teaching) . 

6.2 CAUTIONS 

The above findings must be interpreted with caution. Firsts 
they are not surprising since ^ in general^ conventional 
allocation methods were never intended to promote student satis- 
faction in the first place . Indeed ^ producing teaching that 
interests students is only one aspect of what teachers are 
supposed to produce. As was stated at the outset^ most would 
agree that student achievement is a more important output 
measure. And^ of course^ there are many other outputs (e.g.^ 
student's social skills^ self esteem^ etc.) whose simultaneous 
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maximization with satisfaction^ could strongly effect the 
allocations indicated here. 

Moreover r the sample is limited to just one high school set 
in a middle-class California neighborhood. The results found 
herv^ are not necessarily stable across other high schools; or^ 
for that matter r across specific subject or teacher situations ^ 
or different groups of students. 

Finally r although our dependent variable PYES did let us 
empirically measure student satisf action^ it was far from ideal. 
It does not, for example measure actual satisfaction the way that 
observing actual repeat buying would. Nor is it likely to link 
firmly to the actual student dropout numbers mentioned in Section 
1 (though future research could ameliorate this by administering 
questionnaires to recent dropouts — students who^ in short, have 
chosen not to be repeat buyers of the entire bundle of schooling 
services) . 

But just as Levin and others found that changing salary 
determinants could boost the allocative efficiency of producing 
achievement, the above findings do hint that similar changes 
could improve the efficiency with which schools produce teaching 
that interests students (assuming schools chose that as their 
sole output objective). 

6.3 POLICY IMPLICATIONS 

Since this study's scope is limited by a focus on allocative 
efficiency with respect to student satisfaction only, its 
practical policy implications are also limited — especially 
relative to the bigger picture of how student satisfaction and 
student achievement might interact. Indeed, much of this study's 
value lies in its implications for future research involving this 
bigger picture. 

Nonetheless, the findings here do urge an important policy 
direction: Since the data show student-caring to have far more 
impact on student satisfaction than task-driven does more caring 
(not necessarily more money reallocated to it) should produce 
higher percentages of satisfied students. 

It is hardly surprising that students would care more about 
teaching that cares more about them . Though dismissing so simple 
a conclusion is tempting, in the context of the serious 
disinterest and dropout problems documented earlier, districts 
would do well to heed this empirical linkage, and promote more 
student-caring. 

Policy Specifics 

Precisely how might districts inject more student caring 
into the teaching they offer? The following changes involving 
teacher training institutions, hiring, career ladders, inservice 
programs, and compensation, would help: 

Teacher-Training Institutions : Schools of education could 
emphasize the evidence on the importance of student-caring, 
stressing it's most important trait components (those that 
correlated highest with satisfaction in this study) , and spe- 
cifically how to deliver those traits. They might also impart 
the needec^ skills for that delivery through internships involving 
frequent standardized SET feedback and counseling. 
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Hirihcf : Districts could require standardized internship 
records of SET feedback to help screen for teachers strong in 
student-caring. Further screening could be done via career 
ladders that begin with probationary hiring — with permanent 
employment contingent on SET results (among other measures). 

Career Ladders: Career ladders have already started in 
California and Florida^ largely to offer teachers successive 
steps of position^ pay^ responsibility and status. Such ladders 
could also be implemented to promote teaching that interests 
students. That iSf ladder steps could be implemented, in part, 
as incentives for continued enrichment of key teaching skills. 
And this enrichment could be officially supported and tracked by 
in-service programs involving SETs. 

In-Service Feedback and Enrichment Programs ; Research pre- 
sented in Section 2 suggests that SET feedback can improve 
teaching. Formal programs to regularly measure, plot, and 
counsel on the basis of feedback — feedback from student eval- 
uations, expert teachers, and video tapes — have been successful. 
Indeed, the instrument on which this study is based came from 
such a program. Teacher participation was entirely voluntary; 
plotted improvement in ratings for teachers who truly wanted to 
improve, was substantial. 

Compensation : Besides offering nonpecuniary rewards for 
bonafide improvements in teaching — via personal recognition from 
superiors, formal awards, and all the nonraoney benefits of higher 
steps on the career ladder — serious thought ought to be given to 
revamping teacher salary schedules. These schedules have long 
fostered well-documented problems of insufficient incentives for 
teacher retention, growth, and excellence; low esteem of the 
teaching profession in general; teacher shortages in science and 
math; and allocative inefficiency for producing either 
satisfaction or achievement. 

Three Paychecks 

A practical response to problems of allocative ineffi- 
ciency — and teacher retention, shortages, equity, and incen- 
tives — is offered by Lawler's three paycheck system. In it, 
pay reflects differences between jobs, employees, and perfor- 
mance. Modified to jarve public schools, this system would give 
a teacher a single check written for the total of each of these 
three checks: 

Paycheck 1 : The Job Differences Check, would cover the 
base pay for a given job. It would reflect different 
pay for differences in a) supply and demand factors 
(e.g., math teachers would start at a higher base than 
P.E. teachers); b) workload (more pay to bilingual 
teachers having to prepare lesson plans in both Spanish 
and English rather than just English); and c) other 
competencies and responsibilities not reflected in a) 
or b). 

Paycheck 2: The Employee Differences Check, would 
reflect between- teacher differences in a) length of 
service to the district (this membership reward can be 
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justified since it would save the district on recruit- 
ment and training costs); and^ b) any economically jus- 
tifiable educational attainments beyond those required 
to obtain the job (i.e., beyond those paid for in Pay- 
check 1 ) ; 

Paycheck 3; Performance Differences Check : a bonus 
varying strictly according to performance • As a form 
of merit payc this bonus would be paid only at the mar- 
gin ; the first two checks would form the much larger 
salary base. 

Limitations of the Performance Differences Check 

It is Paycheck 3 that most involves the allocative effi- 
ciency considered here; and Paycheck 3 that is the most prob- 
lematic. This performance differences check represents merit 
pay, plain and simple. And merit pay has not fared well in most 
of the districts that have tried it — thoug.i it has succeeded in a 
few. 

Success of any merit pay plan is critically dependent on 
obtaining objective measures of performance — measures that 
teachers trust. 

Do SETS provide such measures? McKeachie (Section 2) felt 
that although SETs do reflect teacher effectiveness in teaching, 
they are not sufficient evidence of that effectiveness. It 
follows that set's would best serve as just one of several 
performance measures , taken over a sufficiently large sampling of 
classes, and cautiously interpreted by respected peers sensitive 
to the bigger picture of context — the subject matter , students 
involved r etc. Other measures might include classroom 
observation by respected peers> and progress on objectives 
predetermined by the teachers themselves. 

The formidable problems of attempting Paycheck 3, may, in 
part, be reduced by an intriguing extension resembling the 
Scanlon plan.^"^^ Developed by Joseph Scanlon of the United Steel 
Workers Union, "The Scanlon plan overcomes, almost all the 
objections of individual and small group incentive plans and 
generally is considered very successful. Applied to the 
context of schools, the plan would have administrators set aside 
bonus monies in proportion to the extent district effectiveness 
objectives were met overall (e.g., targets of aggregate 
achievement gain scores, reduction in the costs of school 
vandalism, increased teaching that interests students). 

For example, assume that last yearns annual teacher 
payroll of $1 million produced average ratings of 1.0 for 
student-caring. If this year, the same $1 million payroll boosts 
the average to 1.1— and the increase is substantive— the district 
reaps additional "output" for what might otherwise have cost it, 
say, $50,000. That amount would go into a bonus fund to reward 
specific teachers for high bonafide r '.,is in caring. 

Though implementation of such a plan would surely involve 
formidable technical and political problems, a year-end bonus 
fund proportional to the success of the group, would serve two 
ends. It would encourage individuals to boost their output, 
while encouraging ail to work cooperatively with their colleagues 
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as a team# Moreover ^ with a union member as its originator ^ and 
union concerns at its hearty It might even be palatable to 
teachers' unions. 

Unfortunately^ the politics (see Section 2), competencies, 
levels of commitment r and interaction of a vast fabric of other 
interdependent forces makes prediction of any plan's success 
difficult in any context. In short, the particular success of 
the above suggestions cannot be known for a given context until 
it is tried — tried where every effort is made to motivate 
commitment,* agree on goals, work to measure them objectively, and 
reward their attainment irreproachably. A tall order in any 
administrator's book. 

6*4 IMPLICATIONS FOR FUTURE RESEARCH 

As mentioned earlier, much of this study's value lies in its 
implications for future research, especially regarding questions 
prompted by the managerial grid for teachers. Among the serious 
questions that remain are: Precisely where would the all- 
important output " average achievement gains controlling for stu- 
dent background" fall on the grid? And if we change the 
dimensions from meaning "degree of correlation with a factor" to 
mean "degree teacher is task-driven or student-caring," where 
would maximum achievement fall? Does it sit at the respective 
maximums for STUDENT-CARING and TASK-DRIVEN (i.e., the upper 
right hand corner of the plot)? Or do diminishing returns 
operate so that excesses of either factor actually impede 
achievement (placing it closer to the origin along the northeast 
45-degree-angle line)? Or does achievement not correlate as well 
with one factor as it does with another (e.g., placing it further 
out on task-driven, and not so far on student-caring)? To be 
sure, knowing the precise coordinates for the point of maximum 
achievement would establish an important landmark by which to 
navigate. Yet as seen in the rollowing, empirically fitting its 
entire surface would offer still more valuable navigation data. 

Fitting and Superimposing the Achievement Surface 

If the empirical fit of an achievement surface were added 
atop the surface already fitted for satisfaction (see the three- 
dimensional plot in Section 5), the heights of the surfaces above 
the grid floor (measured with standardized scores) would show 
predicted achievement gains and satisfaction, respectively, as 
functions of how much a teacher is student-car ng versus task- 
driven. The relative distances between the two surfaces at 
different grid coordinates would show tradeoffs between satis- 
faction and achievement outputs as functions of different combi- 
nations of student-caring versus task-driven inputs. And, such 
tradeoffs, once quantified, would let us contemplate the 
compromises inherent in trying to maximize both achievement and 
satisfaction simultaneously. 

The above approach is well within reach. For example, just 
as the California Assessment Program (CAP) gathers achievement 
measures statewide, so too — perhaps even on the same test 
instrument — could it gather SET data covering teacher traits and 
student satisfaction. Indeed, just as California monetarily 
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rewards CAP achievement gains, so too could it reward 
satisfaction gains* 

6.5 LOOKING BACK: SIMPLER WAS BETTER 

This study leaves behind it a path strewn with highly 
technical concerns and sophisticated tools; e.g., concerns about: 
1) assumptions violated with a bounded dependent variable; 2) 
whether the independent variables should b^ factors^ proxies, or 
fused groups of original variables; 3) appropriate functions 
including Cobb-Douglas, logit, probit, planar and quadratic forms 
(and related concerns regarding the error of predictions that 
account for diminishing returns versus those that don't; wildly 
negative intercepts and their interpretation; and more). VJe also 
examined statistical controls for effects beyond the teachers 
control, corrections for heteroscedasticity; and, related 
regression algorithms that use a constant term versus those that 
don't. 

Curiously in the end, however, the simplest model amply 
answered the questions we sought concerning this particular 
sample. Surely this speaks well for "simpler is better" and the 
overall robustness of basic least squares regression. More 
important, however, it hints that preoccupation with doing things 
right threatens any research from doing the right things ; ^-g., 
like using complicated logit or probit forms" and missing thaf 
simpler model that well answers the most important questions at 
hand; in this case, that simple planar model that clarifies the 
major forces at work, their magnitude, and direction. 

6.6 IK CLOSING 

Some are certain to object to attempts to boost teaching 
that interests svudents — even when done in concert with trying to 
boost acheivament. But equally certain are dropout trends con- 
firming that if students are not pulled^ challenged and fulfilled 
by schooling, growing numbers of them won't stay around to 
achieve anything. 

The major value of thi.?. study, however, lies not so much in 
the i;!^.tisf action and price coefficients it sought and obtained, 
but rather in the general relationships observed, and more 
important, the simple methodology developed to observe them. 

The managerial grid for teachers helped us conveniently see 
those relationships and question others at a glance. ndeed, by 
offering new ways of seeing, and empirically fitting teacher 
input/output, the grid should help us model more complicated con- 
cepts involving interaction of satisfaction, achievement, 
teaching styles, and any otner plottable schooling variables. It 
should suggest, quantitatively, the consequences of teaching 
style tradeoffs. And it should help articulate preferred styles 
(as did Blake et al. for other professions). 

Above all, the grid should ht^lp us define and achieve teach- 
ing styles that better mix student-caring/task-driven inputs and 
saLisf action/achievement outputs — all to aid a bigger picture: 
that of long-term student wellbeing and reversal of trends that 
prompted this study. 
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tions of three of the author's colleagues at San Diego State 
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August/ September, 1981. Also, Andrew Greeley found results 
similar to Coleman's. Greeley concludes that public schools 
appear to be more successful at educatii-g the affluent while 
Catholic schools do better with the poor. See Andrew 
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'^N.L. Gage, Teacher Effectiveness and Teacher Educa- 
tion , Palo Alto, Ca.: Pacific Books, 1972. 

^^J.A. Centra, "Two Studies on the Utility of Student 
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effect exists when it does not^ can trigger advocacy of an 
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for Psychologists . New York: Holt Rinehart, and Winston, 
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SStotal + MSwithin 
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sures of correlation, see Roger E. Kirk, Experimental 
Design: Procedures for the Behavioral Sciences, Belmont 
California: Wadsworth Publishing Company, Inc., pp. 126-127. 
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^°^D.J Veldman and R.F. Peck^ "Student Teacher Charact- 
eristics from the Pupils' Viewpoint," Journal of Educational 
Psychology t vol, 54^ pp. 346-355^ 1963. 

1 ng 
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that relaxes the orthogonality constraint (thereby permitting 
correlation among factors )r rotating the axes to whatever angle 
will best cluster the variables into factor groupings. For the 
excellent summary of factor analysis from which the above was 
drawn^ see N.H. Nie^ C.H. Hullr J.G. Jenkins^ K. Steinbrenner^ 
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' '"^R.F, Bales r Personality and Interpersonal Behavior » 
New York: Holt^ Rinehartr and Winston^ 1970^ pp. 395-396. 
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1 99 

"^"^The plotting software, Menugraph, was developed by the 
staff at the University Computing Center^ at San Diego State 
University, and was run on a VAX/VMS computer in concert 
with a Cal-Comp plotter. 
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it's smallness relative to the district payments for YEARS 
and EDUNITS hardly justifies its complicating the equation 
(and causing multicolinearity problems). 
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